PCX 



;/ORLD INTELLECTUAL PROPERTV ORGANEATlc. 
International Bureau 




INTCRNATTONAL APPUCATTON PUBUSHED U NDER THE PATENT COOPERAHON TREATY rpr-n 



I (51) Internationa] Patent ClassificaUon ^ 
H04N 7/087, 7/08, 7A2, 7/14 



Al 



(11) International Publication Number: VVO 96/05699 

(43) IntemaUonal PubUcation Dale: 22 February 1996 (22.02.96) 



(21) IntemaUonal AppUcaUon Number: PCT/US95/04620 
I (22) IntemaUonal Filing Date: 14 April 1995 (14.04.95) 



(30) Priority Data: 
08/289,499 



12 August 1994 (12.08.94) US 



(71)AppM^t: AC^^ INC. fUS/US]; Suite 2401, Rockefeller 
center. 1270 Avenue of the Americas, New York, NY 10020 
(US). 

I ^ ^^^^J^'^!^n^^°^- ^ 50th Street. New 

Yoik. NY 10022 (US). FREEMAN. Michael Ja 6-8 Blue 
Sea Lane. Kings Point. NY 1 1024 (US). 

I (74) Agmtsi TUTILE Jon. F4 Dorsey A Whitney. Suite 200 1330 
I Connecticut Avenue. N.W.. Washington. DC 20036 (US) et 



(54) TlUe: 



(81) Designated States: AM. AT. AU. BB. BG, BR BY CA CH 
CN. CZ. DE. DK. EE. ES. FI. GB. GE. HU Jp'kp*^ 
KP. KR. KZ. LK, LR. LT. LU. LV. my^Gi^'S 

UA, UZ, VN, European patent (AT, BE, CH DE DK 
FR, GB, OR, IE. IT, LU. MC, NL PT SE) OAPI mt^; 
^; CF^ CG. a, CM. GA, GN,'S US] S TO 
TG), ARIPO patent (KE, MW. SD, S2. UG). 

Published 

With international search report. 



SIMULCAST OF INTERACTIVE SIGNALS WITO A CONVENTIONAL VTOEO SIGNAL 




(57) Abstract 

It allois'S Sp^SS' u'^S^^^ CTJSr "^^ ""'^^ in the same standam video sig^U bandwidth 

display (187). as'well « nonSl v£Sg ^"J? ^^enS^ZITv^^^ " " 

feature is that interactivity is offered as L q,Su^SZ^ SSS ^^^^ " «^vision sets (186). An important 

mteractive components (600). InteractivityT^n^^^r^ or mtemq,aon m program content to users who do not have the 
be provided as pan of the trirnnit^vKiES^SdSS t^i "'i'^"" "^'"^ """^^ ^^^y^ *a, can 

after subsoiber eatty and/or later at rr^^,^^^^^^"^! ^"^"^ *e subscriber eitheT immediately 

specialiied attention to each intaactivcsu^bJ^ ^ ^'^^^ P™««^ « sul^ly «<> P^sent 



FOR THE PURPOSES OF INFORMATION ONLY 



Codes used to idwitify States party to the PCT on the fnxA fiages of pamphlets publishing international 
applications under the PCT. 



AT 


AostrU 


AU 


Aosnlia 


BB 


Battados 


BE 




BF 


BtafctittFtso 


BG 


Bulgtiii 


BJ 


Benin 


BR 


Braxfl 


BY 


Bchnts 


OA 




OF 


Cesdil Afncn Republic 


CG 


Cooso 


CH 


SwftieritDd 


a 


CAced'Ivoire 


CM 


Cmerooo 


CN 


OUna 


OS 


CttdKMlovakia 


CZ 


Csedi Repobtic 


DE 


Ocnoioy 


DK 


Deonsik 


ES 


Spain 


n 


Finltnd 


FR 


fruoe 


GA 


GaboQ 



GB 


United Kingdom 


GE 


Geoffia 


GN 


GttiDea 


GR 


Oteeoc 


HU 


Huaiaiy 


IE 


betaDd 


IT 




JP 


iaptn 


K£ 


Kenya 


KG 


Kyisyitan 


KP 


Deoocntic Ftople't Republic 




of Korea 


KR 


Republic of Kocea 


KZ 


Kaxakitttan 


U 


LiedttCQStein 


LK 


SriLanlca 


LU 


LBunbourg 


LV 


LMvia 


MC 


Mooaoo 


MD 


RepobUc of Moldova 


MG 


Madafaicar 


ML 


Mall 


MN 


Mongolia 



MR 


Masriunia 


MW 


MaUwi 


NE 


Niger 


NL 


Nedieriands 


NO 


Norway 


NZ 


New Zealand 


PL 


Poland 


PT 


POfiugal 


RO 


Roounia 


RU 


Rtmian Federation 


SD 


Sudan 


8E 


Sweden 


SI 


Slovenia 


SK 


Slovalcia 


SN 


Senegal 


TD 


Chad 


TG 


Togo 


TJ 


Ta^Uftan 


TT 


mUdad and Tobago 


UA 


Uknine 


US 


United States of America 


UZ 


UzbeUsun 


VN 


Viet Nam 



wo 96/05699 



PCT/US95/04620 



SIMULCAST OF INTERACTIVE STCNAT ^ WITH A CONVENTIOWAT. y^ v^r^ 

This is a continuation-in-part of application Serial No. 08/228.355, 
filed April 15, 1994. entitled "A DISTANCE LEARNING SYSTEM 
PROVIDING INDIVIDUAL TELEVISION PARTICIPATION. AUDIO 
RESPONSES. AND MEMORY FOR EVERY STUDENT." The interactive 
television program is of a type disclosed in commonly copending U.S. 
Patent Application Serial No. 07/797,298, filed November 25. 1991, and 
in U.S. Patent Nos. 4,847.700. 4.507,680. 4.573.072. 3.947.792. 
4.602,279, 4.264.925. and 4.264.924. the contents of which are herein 
incorporated by reference. 



BACKGROtlND OF THE INVENTION 
Interactive television system technologies are currently being 
Introduced into the telecommunications scene. These technologies are 
being applied primarily to provide entertainment and educational 
opportunities to the television industry. Interactive capability has been 
developed using one-way systems through the provision of multiple 
parallel channels of information, related in content to one another. For 
example, commonly owned Freeman ct al. patents, U.S. Patent Nos. 
4,264.924 and 4,264,925. disclose interactive televisioirisj^tems-where 
switching among multiple broadcast or cable channels based on viewer 
selections in response to interrogatory messages allows an interactive 
capability. 

These systems have been enhanced to include memory functions 
using computer logic and memory, where selection of system responses 
played to the viewer are based on the processing and storage of 
subscriber responses, as disclosed in Freeman patent. U.S. Patent No. 
4,507.680. 

The benefits of providing interactivity through the use of dififerent 
audio responses was disclosed in Freeman. U.S. Patent Nos. 4,847.698. 
4,847.699, and 4.847.700. Tliese television systems provide a common 
video signal accompanied by several synchronized audio channels to 
provide content related user selectable responses. The audio signals 
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produce dlfiferent audio responses, and in some cases, these are stable 
synched to a first audio script and to the video signal (such as to a person 
or character on the display) providing the perception that the person's or 
character's mouth movements matched the spoken words. 
5 These prior art systems generally provided interactive capability 

through the use of several parallel related infom^tion segments, each 
segment requiring a separate channd* What Is needed is a fully functional 
interactive program which can be normally received by noninteractive 
television receivers as a conventional video broadcast What is needed is a 

10 system not limited to receiving multiple parallel tracks of audio 
transmitted simxiltaneously. What is needed is an interactive ^tem 
where interactivity is not restricted to a question and answer format. 
Because channel capacity is limited in most transmission c^tems today, 
whzt is needed is a system where full interactivity is provided in the same 

15 bandwidth as currently required by a standard television signal. What is 
needed is a system that individualizes the interactive experience through 
personalized graphics video overlay and personalized audio channels. 
What is needed is a ^tem for providing a fully interactive capability to 
every home, regardless of transmission media. What is needed is a 

20 system for simtdcasting both conventional television programming and a 
full interactive program, all within the same signal. 

fiTTM^^^^ OF THE INVENTION 
The present invention can allow each subscriber to a program 

25 received in the home, to receive an individualized and personalized 
eicperience. A truly innovative aspect of this invention is the simulcast of 
a fully interactive program allowing participation by subscribers through 
the use of an interactive program box, and yet viewers without the 
interactive program box can view iminterrupted normal conventional 

30 programming. Both programs are provided by reception of the same 
standard video signal of normal conventional programming. This dual 
functionality can be provided by any one of several possible bandwidth 
efilcient one-way techniques for sending hidden multiple interactive 
audio segments, graphics data and programming control data, the 
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reception and processing of which provides a fully interactive experience 
for subscribers. Increased Interactive flexlblllfy is possible through the 
use of eictemal data storage such as CD ROM which can store additional 
audio and graphics Interactive elements locally. An important feature of 
5 this new invention is that Interactivity is cflFered as an option, without any 
degradation or interruption in program content to users who choose not 
to interact or do not have the interactive program box. 

By creating this more intimate relationship between the viewer and 
convenUonal programs, a novel, exciting kind of television programming 

10 is created. Yet viewing of conventional programs is not affected. 
Individualization to each subscriber comes not only from the 
personalization of audio responses and the memory capability of the 
system but video graphics overla3r8 are also personalized further 
enhancing the interactive experience. Furthermore. interacUve 

15 programming costs are dramatically reduced since fully functional 
interactive elements can now be carried on normal everyday television 
program broadcasts. 

The interactive innovative features of the present invention can be 
provided to participants either with an interactive program box 

20 coimected to a conventional television set or to computer users by way of 
a mtiltimedla personal computer (PC) with an RF demodulator board and 
decoding hardware and software. 

With the present invention, interactivity is personalized preferably 
through the use of alternative audio responses and graphics displays. 

25 which are presented as a function of either a current subscriber remote 
entry or earlier stored subscriber remote entries using the memory 
capability of the system where the system remembers earlier subscriber 
responses and uses these responses in formulating and presenting 
individualized audio and graphics messages to the subscriber. 

30 Personalization of audio can be enhanced through the use of preloaded 
additional audio response segments in the interactive program box and 
stacked audio response segments in time. Further enhancement is 
possible through the use of external data storage devices such as CD ROM 
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for Storing interactive audio and graphics elements* increasing the 
responsive capacities of the system of the present invention^ 

Unlike previously disclosed interactive systems, the present 
invention no longer solely requires that personalized messages be 
5 presented immediately after the subscriber has made an answer choice 
selection. The novel use of trigger points overcomes the limitation of the 
question-interactive answer response formats described in previous 
patents^ ancT appficaUons^y a^ to be 

provided to the subscriber at unknown times throughout the program. 

10 Interactive audio and graphics can show up anytime during the program 
as a function of stored subscriber information. 

This enhanced interactivity is provided through the use of trigger 
points spread throughout the program and identifiable by codes 
embedded into the video signal. Trigger points provide designated times 

15 during the program when the program content is subtlely altered to 
present individual and specialized attention to the particular subscriber 
viewing the television or computer screen. The trigger points are 
essentially markers on the program that effectively trigger macros which 
call for a certain interactive function (e.g.. the playing of a personalized 

20 audio segment and/or the overlay of a graphics screen) to occur using 
stored subscriber selections to questions presented preferably at the 
beginning of the show, or retained firom a previous show, and/or in 
combination with a particular pattern of interactions solicited and 
entered during the program. TTie interactive program box controller 

25 recognizes the trigger points by decoding invisible codes embedded in 
the conventional video signal. The exact timing of the trigger points are 
imknown to the subscriber in order to make the interactive elements 
appear imsollcited to the interactive subscriber. In this manner, the 
interactive elements are presented in a more subtle and less noticeable 

30 fashion thereby increasing signiflcantiy the personalized experience to 
home viewers. 

Meanwhile, subscribers either without the Interactive program 
boxes or with the box but not authorized to receive the particular 
interactive program continue to receive and hear the standard program 



4 



wo 96/05699 



PCT/US9S/04620 



audio and view the conventional video without being offered Interactive 
choices and specialized responses. Therefore, both an Interactive 
program and a nonlnteractive program are being simultaneously 
transmitted to subscribers. 

This high degree of individual participant interactivity is provided 
to the home television or personal computer through the use of only one 
standard video slgnaL A nonnal television signal has an audio carrier and 
a video carrier. Standard audio, synchronized to the video presentation, 
is Inserted into the normal audio carrier, thereby maintaining 
compatibility with any conventional television set In the present 
invention, unused lines of the video are preferably used for embedding 
additional Interactive response audio channels and graphics and control 
data. Alternatively, the Interactive audio segments could be provided 
either serlaUy, one after another. In the audio subcaixler (SAP), or In 
cable frequency guard bands, and/or prestored In memory at the 
Interactive program box. hi addition, even more audio and graphics 
elements can be provided through the use of external storage devices or 
game cartridges. In any event, with the present Invention It Is 
inconsequential whether the Interactive elements are broadcast 
synchronously, serially, on separate channels, embedded In the video, 
transmitted during or before the program, stored In external data 
storage, etc. Any of these elements, no matter where they come from, 
are called up by the Interactive box at designated trigger points through 
the use of overlaid logic sent down In embedded codes In the signal or 
resident In software at the receiver location. In effect, a normal 
conventional television program and an Interactive program are broadcast 
simultaneously from a centralized location to a multitude of subscriber 
homes, some of which are equipped to receive the interactive program 
^^e others are not. 

The enhanced level of Interactivity is provided to interactive 
subscribers in their homes through the use of interactive program boxes 
cormected to television sets, which receive signals either through cable, 
direct broadcast sateUlte. television broadcast, or other transmission 
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means. In the present invenUon. the programming can be provided in 
both analog and digital embodiments. 

It Is the object of the current invention to simulcast an interactive 
television program and a conventional television program, all as part of 
5 one standard video signal to each of a multitude of subscribing interactive 
participants. 

It is an object of the invention to present interactive elements 
throughout the program at times unknown to ttie subscriber. 

It is an object of the invention to deUver instantaneously 
10 personalized messages to subscribers in their homes through the 
presentation of personalized audio responses and graphics. 

It is an object of the invention to use an analog transmission means 
for sending multiple Interactive audio signals and data codes with a single 
video signal. 

15 It is an object of the invention to use digital transmission 

technology for ccnnblning multiple interactive audio signals, a video signal 
and data codes onto a single composite digital interactive video signal. 

It is an object of the Invention to remember earlier subscriber 
selections in tailoring personalized graphics and audio responses. 

20 It is an object of the cturent invention to provide a dual function 

program: a program that is conventional and can be viewed on 
conventional television receiving equipment and also, allow for this same 
program to be received interactively once subscribers have the interactive 
program box. 

25 It is an object of the current invention to recall interactive audio 

and/or graphics elements from an external storage device. 

It is an object of the current invention to create an interactive 
television system that can provide demographic Information to a head- 
end location to be used for survey purposes or for the creation of targeted 
30 commercials. 

These and other advantages, features and objectives of the invention 
and the manner of attaining them will become apparent and the Invention 
will be best understood by reference to the foUowing description of the 
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embodiments of the Invention In conjunction with the accompanying 
drawings and appended claims. 

PWEF DESCRIPTION OF THE DRAWmftS 

Figure 1 Is an overview of the operations center, transmission 
subsystem and home reception subsystem of ^e conventional video 
system. 

Figure 2 is a time diagram showing a representation of trigger 
points and coirespondlng alternative audio segments, one of which is 
selected for presentation to the subscriber Immediately after the 
execution of a trigger point function. 

Figure 3 is a diagram of the preferred embodiment of the 
interactive programming box for extracting the Interactive audio 
segments and data from the conventional video signal bandwidth where 
such interactive elements have been embedded into the composite 
interactive video signal at the operations center. 

Figure 4 is a more detailed schematic of the hardware elements of 
an analog four-channel VBI extiractor in the interactive program box as 
diagrammed in Figure 3. 

Figure 5 is a more detailed schematic of an alternative preferred 
digital embodiment comprising a digital extractor comprising a digital 
decoder/demultiplexer configuration. 

Figure 6 is a diagram of a second embodiment of the interactive 
programming box for extracting and storing the interactive audio 
segments from an SAP audio channel in which the interactive audio 
segments are serially aligned in the SAP channel at the head-end. 

Figure 7 is a diagram of a third embodiment of the interactive 
programming box comprising two tuners, one of which is for tuning to 
and demodulating the standard video and audio signal and the second of 
which is for demodulating an secondary analog carrier comprising 
modulated serial digital audio segments. 

Figure 8 is a diagram of a fourth embodiment of the Interactive 
programming box also comprising two tuners but with a digital 
demultiplexer configuration for demultiplexing the digital audio stream 
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Into n parallel digital audio channels, wherein the n parallel digital audio 
channels are time division multiplexed at the head-end and transmitted 
as a separate digital audio stream. 

Figure 9 is a liittffr^m showing the hardware elements of a preferred 
5 analog transmission embodiment at the operations center comprising an 
Inserter for embedding four audio circuits and data commands into a 

single video signal. 

Flgare 10 is a block diagram showing the hardware elements at the 
operations center of a digital transmission embodiment comprising 
10 <1igi»?i encoder/compressor configurations and a digital multiplexer. 

pye7/iTT.^n p ysscRipnoN o f the preferred embodtmewt 

I. Introduction 

The present invention is a system for simulcasting a fuUy interactive 

15 program allowing subscribers to interact with the program through the 
use of a remote device (604) and an interactive program box (600) 
connected to a conventional television set (186) or through the use of a 
keypad and personal computer (187), with normal conventional 
programming, viewable by anybody with a regular television set (186) or 

20 computer display monitor (187), as shown in Figure 1. 

While such interactive programming may include a plurality of 
video signals, the interactive television programming used herein 
preferably comprises a single standard video and audio television signal 
with a plurality of additional audio signals and/or graphics data for 

25 providing interactivity. The interaction with the subscribers comes 
primarily by way of selection of one or more linked audio segments firom a 
plurality of audio segments, whereby the selected audio segments) are 
chosen as a function of previous user responses. Interactivity is 
enhanced through the use of overlaid graphics displays on the video 

30 which like the audio responses, also vary according to selections made by 
the subscriber on the remote device (604). Audio segments are used to 
provide personalized responses to subscriber selections. The graphics, 
on the other hand, are used to both query the subscriber, preferably at 
the beginning of the program, and also to provide personalized graphical 

8 
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messages to subscribers. The Interactive television program also 
comprises control data for controlling an Interactive program box 
controller (178) at the subscriber home. 

Interactivity is further enhanced In the preferred embodiment 
5 through the application of trigger points scattered at various 
predetermined times throughout the program. The trigger points 
correspond to times when interactive events are scheduled to take place. 
These interactive events could be the selection and playing of audio 
segments or the display of graphics. While the choice of particular audio 

10 segments or graphics is still dependent on viewer selections, the viewer 
selections in response to displayed graphical interrogatoiy messages are 
preferably made during a period at the onset of the program or when a 
viewer first times into the program. These viewer selections are then 
utilized as inputs to macros called up at later times during the program 

15 by the controller upon the occurrence of the trigger points, identified to 
the interactive program box by tmlque codes embedded in the video 
signal. Alternatively, there can be an immediate response firom local 
storage such as RAM. FLASH or ROM. external data storage such as CD- 
. ROM. fix)m the composite interactive signal or from a series of audio 

20 segments that can be collected fi*om both the signal and storage. 
II. Svstem rnmponents 

A Genera] Overview 

As shown in Figure 1, the system uses an Interactive program 
delivery system (98) with any broadcast transmission means Including 

25 satellite (150). cable (138) or television (142) to deliver the simulcast 
interactive and conventional program (hereinafter "composite interactive 
program") fix}m a centralized location, or operations center (608). for 
distribution to subscribers in their homes. The program may be 
broadcast live from the operations center (608). For example, live 

30 sporting events with added interactive elements can be broadcast fi'om 
the operations center (608). However, it is more likely with the present 
invention that the program will be produced off-line and stored In a 
program storage means (133) at the operations center (608). Likewise, 
distribution of the signal at the receive end can be by any suitable 
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transmission means (166) Including a cable distribution system, fiber 
optic, microwave, etc. In the preferred embodiment, the system 
generally Includes: an operations center (608) where the composite 
interactive program is broadcast, the composite interacUve program 

5 comprising a single video signal, a plurality of audio channels, graphics 
and control codes: a broadcast transmission system comprising 
transmission equipment for^transmittlng the toteractlve video sig;n8a by 
ai^ suitable broadcast transmission means and receive equipment for 
distribution to the remote home locations; and at the subscriber home; an 

10 interactive program box (600) for processing the signals and providing 
for the interactivity; a remote device (604) for transmitting to the 
interactive program box (600) subscriber selections; and a standard 
television set (186). Alternatively, the interactive program signal can be 
received, processed and displayed by way of a multimedia personal 

15 computer (187) comprising an RF demodulator board, an extractor board, 
and keypad. 

A major aspect of this invention is the individualization of audio 
response to subscribers In the home. Each participant receives 
personalized audio responses based on one or more selections on a 

20 remote (604) keypad. Since the system can remember earlier subscriber 
selections, the siabscriber selections may be input and stored in memoiy 
at the beginning of the program and used later by the interactive program 
box (600) In formulating an interactive response. The audio response is 
normalfy a prepared prerecorded audio response comprising one or more 

25 audio segments, to the present tovention. not only do the audio 
responses provide interactivity, but the graphics displays also can vary as 
a function of previous user selections thus generating more 
individualization of the program. 

At the remote sites, participants Interact with the interactive 

30 program either via a subscriber interface (604) or computer keypad. 
Preferably, the subscriber toterface (604) is an fafrared (IR) datallnk 
remote device with a keypad or if the interactive program is received by a 
personal computer (187). a PC keypad. Depressing a button on the 
keypad (604) provides Input to the tateractive program box processor 

10 
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(178) which can either Immediately or later be used by the controller to 
trigger trnique algorithmic codes to produce a tmique and Individual 
message from a plurality of transmitted audio segments and/or those 
prestored at the InteracUve program box (600). The response can be 
directed to the subscriber's current answer selection to an interrogatory 
message displayed on the screen, or may be based oii earlier entered 
subscriber selections through the provision of memory and logic in the 
system and trigger points, as described below. 

Multiple audio segments forming the set of suitable responses to an 
interrogatory message are preferably sent as part of a standard video 
signal. There are a number of different ways to effectively forward the 
necessary audio segments for a given interactive event to the interactive 
program box (600). Different embodiments for delivering these audio 
segments include digital and analog transmission embodiments including 
serial or parallel paths, as described below. Hie Interactive elements may 
be broadcast synchronously (alternative responses aligned in time), 
serially, on separate channels, embedded in the existing video and/or 
transmitted before or during the program. Audio segments tagged for a 
given interactive event can be sent to the interactive program boxes 
(600) much earlier than the scheduled event during the program, in 
which case the segments are preferably stored in temporary memory, or 
the segments can be transmitted concurrently with the event With the 
present invention, it makes no difference how the audio segments reach 
the interactive program box (600) as long as they are available for 
selection at the box (600) at the predetermined trigger points. For 
example, the audio segments could also be stored in local external data 
storage (629) such as CD-ROM. 

In the preferred embodiment. Interactive events occur at 
schedtded predetermined trigger points (900). as shown in flgxire 2. 
These points (900) correspond to the times when the conventional 
program content can be altered and personalized for those subscribers 
capable of receiving the Interactive signal. Hie programmer can place 
the trigger points (900) at any time throughout the program. Since the 
trigger points (900) are unknown to the subscriber, the subscriber does 
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not know when they will receive a personalized message. In other words, 
an interactive response can either Immediately follow a corresponding 
user selection made to an interrogatory message or occur at a later time 
corresponding to a trigger point (900). or any combination of the two. Of 
5 covirse, timing of the interactive events should correspond to suitable 
times in the program where branching to interactive elements is sensible 
and^does not dash with the program conteaat of the conventional video 
still displayed on the television (186) or other dlspl£^ monitor. 

As shown in figure 2. at the onset of a trigger point (900). the 

10 controller will select one of several possible audio (or graphic display) 
responses for presentation to the subscriber. As mentioned above and 
shown in figure 2, some of the audio responses may comprise the linking 
of two or more audio segments. 

In combination with the use of trigger points (900). the present 

15 invention allows for the viewer to select certain options at the onset of 
the program to suit the viewers* preferences. For example, if the 
program broadcast is a live sports event at an eaxfy trigger point (900). 
the viewer cotdd be queried as to whether the viewer would prefer to 
receive audio in English, Spanish. French, or perhaps hear the local 

20 announcer instead of the network announcer. Upon the viewer selection, 
the controller branches to the appropriate channel. 

Each trigger point (900) is identified preferably through the 
broadcast of codes sent as part of the composite interactive program 
signal. The codes preferably include, at a minimum, the following 

25 information: (1) header identifying the occtarrence of a trigger point: (2) 
function ID (e.g.. selection of audio responses or graphics responses.etc.): 
and (3) corresponding Interrogatoiy message(s). The first bit sequence 
simply identifies to the controller (178) that a trigger point is about to 
occur. The Amotion ED designates the macro or other set of executable 

30 instructions for the controller (178) to read and interpret to obtain the 
desired result, e.g.. a selected audio response. Alternatively, the data 
codes could Include the actual algorithmic fimction for the controller 
(178) to apply. 
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Upon extraction of the codes by the extractor (174). the controUer 
(178) reads and Interprets the codes and calls from memory (282. 286) 
either a stored message for display on the screen or a particular user 
selectlon(s) designated by the trigger point codes. The user selecUons 
correspond to subscriber answers to a scries of Interrogatory messages 
preferably presented at the beginning of the progcam. After obtaining the 
appropriate user selectlon(s). the controller (178) reads and performs 
the executable instructions using the tiser 8electlon(s) as input(s) in the 
macro algorithm. The result of the algorithm is either a selected audio 
response and/or selected graphics response. The audio response can be 
called from memory if it is prestored. called from, external data storage 
(629), or the controUer (178) can command the switch (620) to branch 
to the particular audio chann el if the response is broadcast concurrently 
with the trigger point. After the selected audio response is played to the 
subscriber, the switch (620) branches back to the standard audio, shown 
at time ta in figure 2. 

As mentioned above, a series of interrogatory messages are 
preferably presented when the subscriber begins watching the interactive 
program. These interrogatory messages can be presented in any one of 
three ways. First, the interrogatory messages can be presented as 
graphics displays overlaid by the interactive program box (600) onto the 
conventional video signal, wherein the graphics data is sent in the 
vertical blanking interval of the composite interactive signal. Second, the 
interrogatory messages are presented as graphics displays as discussed 
above, except the graphics data comes from local storage such as RAM 
(282) or FLASH ROM (286), external data storage (629) (e.g.. CD ROM. 
cartridge, etc.), or a combination of data In the VBI and data called from 
either local (284) or external data storage (629). Hifrd, graphics data 
can be presented in the form of user templates stored at the interactive 
program box (600). 

User selections corresponding to answers to the n successive 
interrogatory messages are received by the remote interface (628) at the 
beglrmlng of the show, stored in memory and used throughout the show 
at the appropriate trigger points to subtlely change program content as 
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the show progresses. Preferably, each interrogatoiy has a set of possible 
answers. Next to each possible answer will be some Identifier 
corresponding to a label on a key on the subscribers remote (604). The 
subscriber depresses the key corresponding to their answer selection. 
5 This selection is decoded by the remote interface (628) and controller 
(178) . stored in memory (282, 286), preferably RAM (282), and used 
later as required by an algorithm designated at a trigger point 

Graphics can be overlaid onto the video at the subscriber site 
throu^ the use of a character generator (624), as shown in figures 3 and 

10 6-8. Hie character generator (624) is essentially an on screen display 
chip. After embedding the graphics codes into the program video signal, 
the microprocessor (178) interprets the codes and maps the codes into 
the actual bit maps, representing the characters, stored in memory (282, 
286). Tlie interactive program box processor (178) then conmiands the 

15 character generator (624) to overlay a particular character designation at 
a particular point in the screen. 

Altemattvefy, graphics video can be sent from the operations center 
(608). The graphics video can be created and output from a personal 
computer (114) to a video mixer (108), as described in Harper et al.*s 

20 U.S. patent application no. 08/228,355. Referring to figure 1, additional 
inputs to the video mixer (108) may include the conventional video from 
either live cameras or VCR. The video mixer (108) outputs a single video 
signal by switching between the graphics video and program video at 
appropriate times. 

25 A plurality of graphics overiscys, each of vMch preferably comprises 

an interrogatory message and wliich are chosen as a function of previous 
subscriber selections in response to previous interrogatories, can be 
provided. The method by which a graphics message is chosen is 
substantially the same as the method used to select audio responses. The 

30 graphics overlays can be selected as interactive responses at trigger 
points as explained in the preceding paragraphs. As with the audio 
responses, one of several possible graphics overlajrs can be presented 
during any given time interval. The graphics message chosen is also 
based on the subscriber selections to displayed interrogatory messages at 



14 



wo 96/05699 



PCT/DS95/04620 



the beginning of the program. The subscriber selections are called by the 
Interactive program box processor (178) which examines these inputs 
according to algorithmic codes to produce the next graphics 
Interrogatoiy message. 

The one*way interactive system also has the advantage of 
remembering subscriber responses and using these responses in 
choosing an audio response, and/or graphics interrogatory message, to 
present to the subscriber. Memoiy branching Is a technique of the 
present Invention where the algorithm assembles audio responses and 
graphics interrogatory messages according to the current and previous 
user Inputs. Memoiy branching is accomplished by linking audio message 
segments or successive graphics Interrogatory messages together in a 
logical relationship. In this scheme, the interactive program box 
processor (178) contains logic (preferably, in the software algorithm) and 
memory (284) to store previous subscriber selections and to process 
these previous responses in the algorithm to control future audio channel 
selection, as well as future graphics message selection. 

H The InteractlYC Prggram Box 

Figure 3 shows an overview of a preferred interactive program box 
(600). Figures 4 and 5 show analog and digital embodiments of the 
audio/data extractor (174), a component shown in figure 3. Alternative 
embodiments of the interactive program box (600) are shown in figures 
6-8. The embodiments represent different apparatus for receiving, 
processing and storing the alternative interactive audio segments which 
are received in different transmission formats. With these embodiments, 
the interactive systems are no longer solety limited to selecting audio 
firom multiple parallel tracks of audio, related in time and content, nor is 
the interactive questlons-lmmediate answer format, as disclosed in 
previous patents, necessary. Of course, the systexhs of the present 
invention can still use the question-immediate answer format or a 
combination of such format and delayed response via trigger points. The 
concept remains the same, i«e., to select audio responses which are 
matched to user selections by some function. However, the current 
Invention, as supported by the disclosed embodiments, goes farther to 
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enhance interactivity by providing the interactive elements at times 
imknown to the viewer. 

The interactive program box (600) Is a portion of the system that 
can reside in the home of the subscriber or elseixiiere. such as at a cable 
5 headend, as described below. If at the home, the interactive program box 
(600) Is usually located near the subscribers' television (186). but it may 
be placed any\<^ere within the range of the subscribers remote handset 
~^64)7 Assho^ ngure 37the pi^ box (600) 

comprises an RF demodulator (616). comprising video and analog 

10 demodulators, an audio switch (620), voice/data extractor (174). modem 
(312). audio memory (702) a controller (178) with associated memory 
(284). Gen-lock circuit (623). character generator (624). an RF 
modulator (622) and an inficared Interface (628) for the subscriber 
remotes (604). An external data storage device (629) and printer (302) 

15 are optional. 

iatematively. the elements of the interactive program box (600) 
can be incorporated and provided t:>y a conventional multi-media personal 
computer (187). Preferably, this computer preferably comprises a video 
demodulator board, a keypad for entering subscriber selections instead of 

20 a remote IR device (604). an extractor board for separating the audio 
signals and data from the conventional video signal, temporary and 
permanent data storage, a modem (optional), audio switch and a 
processor. Operation of the interactive program system using a personal 
computer (187) Is substantially Identical to that of the interacUve 

25 program box (600)/television set (186) combination described below. 

In addition, although the interactive program box (600) is most 
likely to be located in the home, there is no reason why the interactive 
program box (600). or a functional substitute (comprising a processor, 
extractor, memory, audio switch. RF demodulator, etc.). could not reside 

30 at an location external to the home, such as at a cable headend. If the 
interactive program box (600) is located at the headend, the interactive 
program box (600). or equivalent, could receive user selections from the 
home via a wireless RF link, telephone line, cable, etc. Once user 
selections are received, selection of the audio and graphics signals at the 
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cable headend occurs In Identical fashion as If the Interactive program 
box (600) were located In the home, as described below. The path to and 
from the cable headend can be seen as simply an extension of the cable 
betweai the Interactive program box (600) and the television set (186). 
The only possible distinction Is that one Interactive program box (600) 
located at the headend may be programmed to sekvlce several subscriber 
homes eflfectively using a time sharing box among maxiy users. 

fa a preferred embodiment, the plurality of audio signals are 
combined \wlth the single video signal along with interactive data and 
graphics codes in an inserter (130) at the operations center to form a 
single NTSC compatible video signal. These signals may be analog or 
digital, as described below. 

Referring to a preferred embodiment shown In figure 3, the 
composite interactive signal Is received by the Interactive program box 
(600) either via cable coimection. If the signal Is transmitted over cable, 
by way of satellite antenna hook-up for receiving sateUlte broadcasts, or by 
way of a television antenna If the signal is broadcast The composite 
interactive signal is input into an RF demodulator (616). fa the standard 
television mode with no toteractivlty. the RF demodulator (616) simply 
demodulates the conventional video and audio signal and outputs the 
standard audio which passes through the audio switch (620) to the 
television set (186). The video Is forwarded to an RF modulator (622) for 
presentation on the television or computer screen. Therefore, in the 
standard mode the toteractive program box (600) is simply a pass- 
through box. The voice/data extractor (174) and audio switch (620) are 
completely bypassed in this embodiment. 

fa the Interactive mode, the video demodulator (616) outputs the 
standard video signal which Is transported to a Gen-lock circuit (623) 
and character generator (624) as wells as to a voice/data extractor (174). 
At the output of the Gen-Lock circuit (623) and character generator 
(624). the video Is forwarded via the RF modulator (622) to the television 
(186) or computer display monitor. The processor (178) preferably 
controls an n X 1 switch (620). the output of which Is an appropriate 
audio segment to be sent to the television set (186) for presentation to 
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the subscriber. Of course, the switch could have more than one output In 
\^ch case more than one viewer can watch the video on the same 
monitor and each receives Individualized audio response through the use 
of headphones. The processor (178) sends a command to the audio 

5 switch (620) to discoimect the standard audio at the beginning of an 
interactive segment The extractor (174) essentially reverses the process 
by-^di the audio^and. data signals were inserted Into the video ^gnal. 
As explained below, the voice/data extractor (174) removes the additional 
audio segments and data that are hidden in the standard video signal. 

10 Hie data is forwarded to the microprocessor (178) and the audio 
segments get sent either to an audio switch (620) or to temporary 
memory (202) depending on vrtiere the instructions teach the segments 
to be forwarded, all of which occurs under the control of the 
microprocessor (178). The microprocessor (178) reads and interprets 

15 the instructions either broadcast in the data codes or resident in the 
operating software at the interactive program box (600). 

The microprocessor (178) Interprets the extracted data as either 
control data, including instructions for switching between voice chaimels. 
or graphics data for on screen display. If the data is on-screen display 

20 data, the data is preferably prefixed by a command designating the data as 
on-screen display data, as opposed to control data. In the preferred 
embodiment the controller (178) also examines the control data for the 
occurrence of a header code designating the onset of a trigger point in 
the program. 

25 If the trigger point codes designate a macro vrbich calls for the 

placement of a graphics display on the video, the microprocessor (178) 
reads the codes, accepts ar^ graphics data sent from the head-end (608), 
calls for and examines the actual bit maps stored in memory (284) or 
external memory (629) and designating the identity of the characters. 

30 and then commands the character generator (624) to overlay particular 
characters at particular points on the screen. Therefore, the graphics are 
preferably generated locally with the bit maps stored in memory (284). 
The graphics are selected for presentation either in predetermined 
sequence, through the use of control codes in the composite interactive 
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program, developed when the program was created at the operations 
center (608). or more flexibly through the execution of algorithms by the 
processor (178) utilizing stored subscriber selections to previous graphic 
Interrogatory messages. The algorithms are preferably part of the 
5 operating systems software stored In memoiy at the interactive program 
box (600). Alternatively, the algorithms could be Included In the data 
portion of the composite Interactive signal. 

The graphics can be utilized to overlay any portion of the screen of 
the television screen. The character generator (624) is locked by a Gen- 

10 lock circuit (623) v^ch allows for the synchronous placement of the 
graphics on the video. The character generator (624) is preferabty a 
standard on-screen display chip which takes incoming video, locks the 
video and superimposes on the video the characters as instructed by the 
microprocessor (178). Specifically, the character generator (624) is a 

15 switching system which takes the active lines of video and switches to a 
mode of sending the graphics characters for a predetermined time, and 
then switches back to the video when the character is finished being 
written on the screen. 

Because the graphics are generated locally, subscribers without the 

20 interactive program box (600) are not be able to view the graphics. For 
those subscribers possessing the Interactive capability, the graphics can 
be used for both posing Interrogatory questions to subscribers at the 
onset of the program, consistent with the trigger point embodiment, 
posing questions during the program, or used to provide a personalized 

25 response to previous individual subscriber selections. 

Preferably at the beginning of the program or when a viewer first 
tunes in, a series of interrogatory messages are presented to the 
subscriber. The subscriber responds to the interrogatoiy message by 
depressing a button on a remote (604) or computer keypad 

30 corresponding to an answer selection listed on the interrogatoiy graphics 
screen. If the subscriber has made a selection using a remote (604). a 
signal is received by the JR interface (628) which processes the signal 
and forwards the signal to the processor (178). Hie processor (178) 
preferably creates a packet comprising the user selection and a header 
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code that identifies the particular Interrogatory message associated with 
user selection and sends the packet to memory (284). Bach user 
selection to each interrogatory Is stored In this feshlon. These selections 
will be called later In the program at tq)propnate times when Identified 
by the trigger point codes and then used In macros or algorithms to 
determine Interactive audio and/or graphics responses. 

Hie presentatton of toe grapWcs to messages can also be 

made a function of subscriber selections to previous toterrogatory 
messages. The logic used to the codes for selecting the next graphics 
message Is similar to that used for selecting audio messages. One 
method, as disclosed to earlier ACTV patents. Is the "decision tree" logic 
methodology. Hie subscriber makes a selection to a first predetermtoed 
toterrogatory graphics message. After the subscriber hears an 
approprlatety branched audio channel, the processor (178) will toterpret 
graphics algorithmic codes sent down firom the operations center (608) 
and will read fix>m memory (284) an appropriate next graphics message. 
The processor (178) then directs the character generator (624) to 
overlay the selected graphics message onto the next firames of video. 

Tlie voice/data extractor (174) of the preferred embodiment shown 
to figure 3 is shown to more detail to figure 4. This particular extractor 
corresponds to the Pulse Amplitude Modulation (RAM) tosertcr (219) 
discussed below to connection with Figure 9. The audio and data is 
stripped out of the baseband signal to the four-channel VBI extractor 
(174). The signal separator (174) essentially reverses the process by 
which the audio and data signals were Inserted toto the video signal. A 
video digltiztag system and timing controller (320) digitize the tocomtog 
video signal at tiie same sampltog rate at which the PAM audio samples 
are inserted toto the video signal. Alternatively. If a digital adaptive pulse 
code modulation (ADPCM) exti-actor. corresponding to an ADPCM 
inserter described below, is used to the toteractive system, the extractor 
does not toclude the video digitizing and buffer circuitry shown to Rgure 
4. 

Tlie t«T«in£ conti-oUer (320). shown to Figure 4. determtaes which 
fines of video contata the audio signals and separates tiie compressed 
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audio signals by directing each audio channel Into an associated FIFO pair 
(328-356). As with the Insertion circuit the FIFO's in the FIFO pairs 
(328-356) are used alternately so that while one FIFO is reading the audio 
data, the other is writing data. The digital-to-analog (D/A) converter 
5 (360) sequentially reads from the FIFO for each channel which is in the 
write mode and converts the digital signal for that channel Into an audio 
signal. The analog output for each chazmel is filtered by filters (364-376) 
and directed to bxiffers (380-392). The buffers (380-392) preferably 
adjust and amplify the analog output signals to -IV to +1V range. TTie 

10 four output audio signals correspond with the four audio Input chaxmels. 

As shown in Figure 4. the line 21 data is read into 8-blt shift 
registers (400) wherein the 16 bits of serial data are converted to two 
bytes of data. The line 22 data is read by 8-blt shift registers (402) and 
the 16 bits of serial data are also converted to two bytes of data. 

15 Hie line 21 and 22 data (I.e.. the data codes) are directed from the 

extractor (174) into the microprocessor (178). The microprocessor 
(178) decodes the data and stores It in either RAM (282) or ROM (286). 

In an alternative voice/data digital extractor embodiment at the 
receive end. the composite digital signal is received where the FEC code 

20 is appUed by an FEC decoder (516). as shown In Figure 5. The resulting 
signal is applied to a digital demultiplexer (520) where the audio, data 
and video components are separated and individually buffered In a set of 
buffers (524-532). Ftom buffer A (524). the composite digital audio 
signal enters a digital-to-analog converter (536) where the plurality of 

25 audio signals (Including the standard audio channel and the additional 
audio channels) are separated, decoded and decompressed to recreate 
each individual analog audio channel. From buffer B. the digital video 
signal slmUarly enters a digital-to-analog converter (540) where the 
signal is decompressed and decoded to form an analog video signal. The 

30 data is similarly decompressed and decoded in analog-to digital decoder 
C (544). 

Referring back to figure 3, the processor (178) receives data from 
the extractor (174). stores data and reads instructions from memory such 
as temporary RAM (282) memory, ROM or FLASH ROM (286) memory. 
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and receives response data from the remotes (604) via an infrared 
interface (262). or alternatively a computer keyboard. Optionally, the 
processor (178) can Interface to a printer (302) via a printer port (302) 
for printing out coupon codes, as discussed below. The processor (178) 
5 is preferably a 80C188 based microcontroUer, but can be any similar 
controller. As mentioned above, this entire system could be built into a 
personal computer using a standard off the shelf multimedia personal 
computer (187) with preferably an 80386 or higher based processor. 

The operating system software for the processor (178) is preferably 
10 stored in FLASH ROM memory (286). The FLASH memory (286). as 
shown In Figure 4. also preferably stores new instructions to use with 
reprogrammable software. It works In conjunction with RAM (282). The 
RAM (282) can also be used as an interfiacc to a print buffer. The print 
bu£kr is used so that data transmissions to the site can be buffered, error 
15 corrected and formatted for printing. The site controUer (178) is 
preferably connected to a printer via a parallel printer port. The 
controller (178) sends cottons to the printer (302). 

The interactive program box (600) coxmects with the remotes 
(604) preferably by infrared (DR) connection. The processor (178) 
20 preferably stores the setup commands and interactive branching 
commands and algorithms In associated memory, as described below. In 
the present invention, code interpretation and switching Is preferably 
accomplished by the Interactive program box processor (178). The 
processor (178) collects subscriber responses from the remotes (604). or 

25 computer keypad. 

The advantages discussed above in relation to presenting an 
interactive program using trigger points are obtainable to each of the 
tateractive program box embodiments shown to figures 6-8. In the 
embodiment shown to figure 6. alternative audio segments arc preferably 

30 sent serially from the operations center (608) to the SAP channel. The 
demodulator (617) receives a composite tateractive signal comprising the 
standard video and standard audio signal along with an audio subcarrier. 
The demodulator (617) breaks tiie signal tato it's component parts, 
forwarding tfie baseband video to a data extractor (175) and the standard 
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audio to an audio switch (620). The line 21 data extractor (175) takes 
out the data codes. Including the trigger points. 

The SAP channel comprises a plurality of audio segments lined up 
serially. The audio segments are digitized to the analog to digital 
5 converter (175) and are preferably stored to digital audio memoiy (283). 
At certato times during the program, data codes wlU designate a trigger 
potot and key the microprocessor (178) to select and play an audio 
segment correspondtog to previous user toput(s). accordtog to the 
process described above. The microprocessor (178) calls the appropriate 

10 audio 8egment(s) from totemal memoiy or external data storage (629) 
and commands the audio switch to pass the selected audio segment to 
the RF modulator (622) for play to the subscriber. At the end of the 
toteractlve time period, the controller (178) Instructs the audio switch 
(620) to agato pick up the standard audio. 

15 to an alternative embodiment similar to that as shown to Figure 6 

and discussed above, the simple addition of a second tuner, receiving the 
composite RF signal, could be used to tune to a second audio channel for 
collection of transmitted audio segments. The tuner would pass the audio 
segments to the A/D converter (175) with the operation of the rest of the 

20 toteractlve program box (600) similar to that described above to 
connection with Figure 6. 

Figure 7 shows another toteractlve program box (600) embodiment 
for provldtog alternative audio and graphics segments. This embodiment 
uses two tuners: an RF demodulator (616) and a data tuner (615). The 

25 RF demodulator (616) tunes to and demodulates the conventional video 
and audio sig^ to the standard video bandwidth. The data tuner (615) 
receives a single digital audio slgnaL The signal comprises digital serial 
audio segments modulated onto an analog carrier. The data tuner (615) 
demodulates the signal toto digital audio. The digital toterfiace selector 

30 and error corrector (177) separates the audio segments and performs 
error correction accordtog to any error correction scheme commonly 
understood to the art The controller (178) directs the selector (177) to 
extract selected digital audio segments from the serial digital stream and 
send them to the digital audio memory (283). Selection of one or more 
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audio segments for play as personalized messages on the television set 
(186) occurs according to the processes described above* After the 
controller commands the memory (283) to forward a digital audio 
segment* the segment is converted to analog by the digital to audio 
5 converter (176) and is subsequent^ passed to the RF modulator (622) for 
play on the television set (186)« 

/jio^a: Interactive program box (600) embodiment for receiving, 
storing and selecting alternative audio segments is shown in figure 8. At 
the operations center (608) * the audio segments are digitized, time 
10 division multiplexed, modulated and converted to firequendes in unused 
channel frequency space in the cable television spectrum, e.g., cable 
guard bands. 

The RF demodulator (616) again demodulates the conventional 
video and audio signal. The data extractor (175) receives the signal and 

15 extracts the VBI line 21 data codes. The data in the VBI indicates the 
frequency channels in which the digital audio segments are transmitted. 
For example, audio messages A-E are located in between ch a nn els 14 and 
15. The controller (178) instructs the data tuner (615) to tune to that 
part of the spectrum between channels 14 and 15. Alternatively, an 

20 autotime capability^ can be used to find the audio channels in the 
spectrum. 

The tuner (615) demodulates the digital audio signal and forwards 
the signal to the digital demultiplexer (700). The demultiplexer (700) 
demultiplexes the signal Into n digital audio channels and forwards each 

25 channel to a separate D/A converter (702-710) where each of the digital 
channels are converted to analog audio. As described above, one of these 
rhnnnrffi can be selected as identified at the trigger points for play over 
the audio speaker to the subscriber. All of the other components in 
Figure 8 operate identically to that previously described in connection 

30 with figure 3 to provide interactive or conventional programming to the 

subscriber in the home. 

The embodiments described above and shown in connection with 
figures 3-8 relate to dlfierent ways of receiving broadcast audio segments. 
Alternatively, interactive audio segments, or graphics elements, could be 
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prestored on cartridge. CD ROM or even floppy disk. However, the 
trigger points are preferably broadcast as data codes embedded in the 
conventional video signal and wiU work with any of the embodiments 
described above. 

In this manner, each subscriber at dlfiferent homes hears a different 
and personalized response from their own television set (186) due to 
their Interaction with the Interactive program box (600). As discussed 
below, the power of this system can be Increased exponentially via 
preloading of additional audio segments expanding personalized deliveiy 
and the mecha ni s m of memoxy branching xdiereln the system actually 
remembers and uses the subscriber's earlier selections to present or 
compose unique messages to the subscriber. 

Hie range of possible audio responses can be dramatically increased 
by preloading audio segments. First, more than simply the audio 
response segments broadcast along with a trigger point can be 
transmitted by preloading the memory at the interactive program box 
(600). Preloading can be accomplished by sending additional audio 
segments in a time period prior to that period in which the normal 
broadcast audio responses are sent. 

For example, at the beginning of a program several audio segments 
can be broadcast to the interactive program boxes (600) by any of the 
methods and systems described in connection with figures 3-8. The 
controUer (178) directs that these segments be stored in memory. Each 
audio segment has a pointer which identifies the particular interrogatory 
message corresponding to an audio segment. For example, if the 
Interrogatory message presented early in the program was the following, 
"Are you a boy or a girl." an audio segment which might correspond to 
this question is "Oh. I see, you are a boy." At the time of the interactive 
element, identified to the controller by the trigger point codes, 
preferably more audio responses associated with the earlier presented 
Interrogatory message are output from the operations center (608). 
passed through an audio mixer (118), embedded in the video signal at the 
Inserter (130) and transmitted to the remote sites. Once received at the 
interactive program box (600), the audio responses are extracted from 
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the video signal and forwarded to the switch (620). The controller (178) 
then either selects an audio response made of prestored audio segments 
In memoiy or one of the audio segments broadcast with the trigger point 
codes. Ihe switch (620) branches to the appropriate ch a nn el to forward 
5 either one of the segments from memory or one of the recentty broadcast 
segments* At the end of the interactive period, the switch (620) 
branches back to the standard audio. 

Even more enhanced and flerfble operation can occur ^ough the 
addition of external data storage (629). such as CD ROM or cartridge. For 

10 example, sports statistics or other information on athletes or others can 
be stored In CD ROM. During live sports event either audio segments or 
graphics displays focusing on the athlete can be called by the processor 
(178) and presented to the viewer as a function of user selection of an 
option or at a trigger point if the user indicated during queries at the 

15 beginning of the live event that they were interested in a particular 
player. 

In the present invention, the Interactive audio branching is 
preferably performed in the interactive program box (600). The 
branching algorithm, preferably defined by a series of branch codes, can 

20 be inserted as part of the program. The processor (178) stores the 
algorithm in associated RAM memory (282). The algorithm determines 
the proper channel to switch to as a function of the subscriber selection. 

Hae branching algorithm can be expanded by utilizing previous user 
selections in making a decision on a current audio segment. This is the 

25 memory branching embodiment. In other words, previous user 
responses, stored in RAM memory (282) in the interactive program box 
(600). are used as input to a logic network, or algorithm. Successive 
audio output segments can be related in a *decislon-tree* type 
relationship. Tlie logic network, or algorithm, processes the data input 

30 for selection of an audio output response message. This memory 
embodiment goes well with the expanded audio response capability 
described above. 
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The memory branching technique, described above, can work 
similarly for basing successive graphics Interrogatory messages on 
previous subscriber selections to earlier graphics Interrogatory messages. 

Another embodiment uses -stacking" to increase the audio answer 
response options. The expansion is due to linking separate audio 
segments separated in time to form a greater number of answer 
responses. Hie foUowlng example lUustrates the use of stacking as well 
as memory branching, described above. The left hand column of the 
ecample shows the Interactions between the program and the subscriber. 
Hie right hand column shows the logic applied and branching performed 
hi the Interactive program box (600). 



TypEor 

1. Can you ten time? 

(1) IfES 

(2) 

(3) 

(4) NO 



RETURN FROM BRANCH 

2. WhatttmeJstt? 

(1) 2:30 

(2) 3:30 

(3) 3:50 

(4) 4fi0 



NOIE: 3:30 (button (2)) is the coiTcct answer. 



• Store button pressed in xnemozy A 

• IfbuttODPl.gototzack 1 

• IfbuttODs2.gototzBCk3 

• ffbuttons3.gototiack3 

• irbutton;s4,gototi:ack2 

• If no button pressed, go to track 3 

GOTOTRACKl 

• Store button pressed in mcmotyB 

• ffbuttOQsl.gototiBck 1 

• irbuttim?i2,gototZBck2 

• ffbutton;s3,gototrack3 

• tfbuttODB4,gototZBdc4 

• If no button pressed, go to tzack 4 



Recall response from question 1 
"n^ACKl: "You said you Imow 
how to ten time • • / 
TRACK2: "Ybu said you dont 
know how to tcU time 
TRACKS: TTou didnt indicate if 
you can ten time../ 

Memory branch 

?RACK1: •...andyouwcrent 

kidding. Great Workl* 

TOACK2: •...butyousureleamed 

fast Thafs great* 

TSACK3: and with more work. 

you will leam more." 

•...but with more work, 
you will leam more.* 

RETURN FROM BRANCH 



• Recall memory A and branch as foUows: 

• IfmemotyAsl,gototrad(l 

• If memory Aff4. go to t r^M 2 

• IfmemoryA;s2or3,gototiack3 

• Branch as follows: 

• ffmrmory Agl and Tn^nCT y p=2 
(oonrectlgototrackl 

• ff memory A^ftl and mcmoiyB=2 

(correct) go to track 2 

• tf mem ory Asl andmemoiy 3=1. 3 
or 4 Uruxnrcct) go to track 3 

• ff memory A^l and memory B=l. 3 
or 4 (Incorrect) go to track 4 

GOTOTRACKl 
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Stacking is evident in the above example by the Increased ntunber 
of optional audio messages formed by linking the later two branches 
indicated in the example above. The possible branch combinations are 
5 provided, as follows: 

Branch #1 to Branch #1 

Branch #2 to Branch #2 

Branch #3 to Branch #2 

Branch #1 to Branch #3 
10 Branch #2 to Branch #4 

Branch #3 to Branch #4 

Therefore, the present invention covers nxmierous methods for 
increasing the interactive audio response capabilities. 
C Subscriber Interface 
15 Each subscriber interacts with the system via a subscriber interface 

(or •remote") (604) or a computer keyboard. 

The keypad (604) preferably comprises a number of functional 
keys. The keypad (604) preferably comprises keys labelled 0-10, "YES" 
and "NO" labelled keys, and "TRUE" and "FALSE" labelled keys can be 
20 accommodated by the present invention. The invention can also 
accommodate an LCD panel for receiving messages and confirming 
answers. Similar functions can be assigned to computer keyboard keys. 

Addressability and authorization occur preferabfy losing techniques 
well known in the cable industiy. For example, program selection and 
25 authorization can occur using menus on the screen with ciKsor overlajrs. 
Through the menu system, a viewer can access a program by inserting a 
unique ID code, billing number, etc. Macro access algorithms stored in 
memory in the interactive program box enable entry. 

Either at the begiimlng of the interactive program, as in the 
30 preferred embodiment or alternatively as the program progresses, the 
subscribers are requested to enter responses via the keypad (604) or 
computer keyboard. These requests are preferably displayed on the TV 
monitor (186) or computer screen through the use of graphics overlays. 
The interactive programming preferably consists of graphics 
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interrogatory messages with correct and Incorrect answers overlaid on 
the video. 

The kQrpad (604) or computer keyboard allows the subscriber to 
Interface with the interactive programming by depressing keys 
5 corresponding to answers of an Interrogatory message. For example, in a 
multiple-choice question mode, each participant selects a keypad button 
corresponding to his or her choice. These selections are preferably 
stored and later used by the controller (178) in selectlttg either an audio 
or graphics message and either directing the audio switch (620) to 

10 switch from the standard audio channel to an alternative audio chaimel 
for passing the respective audio segments to the television (186) or to 
direct the graphics character generator (624) to overlay graphics 
characters over predetermined portions of the screen. The user 
selection is mapped Into one or the plurality of audio responses. 

15 comprising one or a combination of linked audio segments, or one of a 
plurality of graphics messages after the user selection is applied to a 
some functional algorithm. 

TX Program Prapflrfltlon 

For the present invention, it is anticipated that the majority of the 
20 Interactive programs will be produced off-line and thereafter recorded 
for subsequent broadcast on some storage medium at a central location. 
Alternatively, live programs could be broadcast with the interactive 
elements Inserted in similar fiaishion to those techniques described in the 
parent case. 

25 The broadcast or operations center (608) is where the task of 

either recording and broadcasting or creating and packaging the 
composite Interactive programs is preferably performed. These 
programs are created to be received and viewed as both regular 
conventional television programs and as an interactive programs. 

30 Therefore, they must be created in such a manner that there is no 
noticeable difference to those nonlnteractlve subscribers watching the 
conventional program when the interactive portions begin for authorized 
interactive subscribers. Likewise, when the interactive portions end and 
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both groups of viewers again view the same video and hear the standard 
audio, there should be no noticeable transition. 

the composite interactive programs that can be created include 
educational programs, home shopping programs, movies, children's 
5 programs, news programs, etc. The stored programs may be in analog or 
digital form and stored in permanent or volatile memory sources. In 
addition. Uve programs can be_ enhanced with interactive elements and 
broadcast to subscriber homes as part of the present invention. 

The interactive program used in the present invention preferably 

10 comprises a single video, a standard audio chazmel. graphics and data 
codes, and a plurality of audio signals embedded therein to achieve 
intcractivily. However, as described and shown above In connection with 
figures 3-8, the audio can be sent on separate channels, preloaded at the 
interactive program box (600), or stored separately in external data 

15 storage. 

Any of these embodiments wHl work with the interactive trigger 
points of the current Invention. The program must first be produced In a 
studio. The resulting program will conq)ri8e the conventional video and 
audio receivable on any television set (186). Next, the desired interactive 
20 sequences must be defined. The producer must decide which porttons of 
the program will be interactive. Tliese portions wlU be Identlfled by 
trigger points. These trigger points correspond to the times ^en the 
program can be altered. The trigger points are essentially codes that will 
call macros that wlU be used to trigger the occiarrence of each interactive 

25 sequence during the program. 

Therefore, the producer must decide at what points in the program 
to include the interactive responses, i.e. the trigger points. The 
interrogatory messages reqtiired for soliciting viewer interests and 
answers must also be scripted. The alternative audio and graphics 

30 responses to possible viewer selections are also created by the producers. 
Next, the interrogatory messages, with possible system responses, must 
be mapped Into predetermined trigger points. This mapping is made 
possible by developing an algorithm on a personal computer. The 
algorithm maps specific system responses (audio or graphic) to occur at 
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the trigger points as a function of viewer selectlonCs). For example, if the 
user selects A to Interrogatory #2. a predetermined audio or graphics 
response Is selected to play at an Identified trigger point. 

Next, software In a personal computer complies the above mapping 
into program transmission codes. The personal computer then controls 
the loading of the audio and data into the program stream at the 
appropriate times and Inserts the codes Into fields. Timing and control 
between the video and multiple audio elements, discussed above, are 
provided through the use of data codes. The data codes are stored in 
memory in a personal computer as part of the ACTV programming 
language. The codes comprise commands, or branch codes, for 
branching between channels (as discussed In the aforementioned ACTV 
patents and applications), timing signals for controlling the interactive 
program, codes deflntog each trigger point data or text, commands for 
termination and initiation of interactive programming viewing, or triggers 
for ocecuting macros. 

Finally, the operating system software must be developed and 
loaded into the interactive program box (600). This software can be 
uploaded to the box on the cable and stored in RAM memory (282). 
loaded onto a cartridge or placed in ROM memory (286). Tbls software 
comprises the macros that interpret subscriber inputs and map these 
Inputs into specific interactive elements at a trigger point during the 
program. 

At this point the audio segments can be stored in a multichannel 
tape storage device or digital device for later use. When it is time for 
broadcast the video and standard audio pass to an inserter (130) to be 
combined with the data codes and multiple audio charmels at the 
appropriate time as commanded by a processor. As shown in ngure 1. 
the audio channels are preferably fed from the personal computer (1 14) 
into an audio mixer (118). Furthermore, this processing can be utilized 
to align the different audio response tracks In time. 

TTie analog inserter (214) of a preferred embodiment is shown in 
more detail in Figure 9. The four audio ou^ut channels are fed into the 
Inserter (214) to be mixed with the video and data. Each audio signal is 
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preferably an analog signal. A bu£fer (222) is provided for each audio 
signal to electrically isolate the audio inserter (214) from the source of 
the audio signals. An amplifier amplifies the signals in order to converts 
the analog audio signals to a format -which is fully compatible with the 
5 analog-to-digital (A/D) converter (234) discussed below. 

The output of each bufiier (222) is directed io a bank of low pass 
filters (226). one for each audio signal. In addition to reducing high 
firequency noise from the audio signal, the low pass filter (226) dips the 
audio signal to a desired firequency. The filtered audio output signals of 

10 the low pass filters (226) are directed as inputs to a multiplexer (230). 

In a preferred embodiment, the multiplexer (230) is controlled by 
timing signals received from the timing circuit (254). The timing circuit 
(254) is synchronized with the video signal of the interactive program 
and controls the multiplexer (230) to sequentially take a sample fiY>m 

15 each audio signal and to direct the sample to an analog-to-digital (A/D) 
converter (234). The A/D converter (234) converts the samples of the 
four audio signals to an 8-btt digital format 

As shown in Figure 9. each of the four audio channels preferabty has 
an associated first-ln first-out (FIFO) pair (238. 242). The 8-bit digital 

20 samples for each signal are directed as an input to one of the FIFO pair 
(238. 242) associated with that signal. The samples for each channel are 
preferably directed to one of the FIFO pair (238. 242) for that channel 
tmtil enough samples are in that FIFO to fill the desired number of lines 
of the video field into which the sampled audio is to be inserted. The 

25 audio samples are then directed to the other FIFO of the FIFO pair (238. 
242) for that channel until that FIFO has the correct number of samples. 
In the meantime, the contents of the first FIFO are written onto the video 
signal. It is foreseen that other types of memory means may replace the 
FIFO pair (238. 242). 

30 When the content of a particular FIFO is to be inserted into the 

video, the digitized audio samples stored In that FIFO are directed (first- 
in-first-out) to a digital-to-analog (D/A) converter (246). as shown in 
Figure 9. The D/A converter (246) converts the stream of digitized audio 
samples into an analog RAM (pulse amplitude modulated) signal. A 
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second bu£fer (250) Is provided for electrtcally Isolating the inserter 
Circuit from the switching circuitry (258) which multiplexes the audio 
signals into the video signaL 

In a preferred embodiment the audio signals A and B are each 
inserted independentiy onto 5 lines of one field of the video signal, and 
audio signals C and D are Inserted Independently onto 5 lines of the other 
field of the video signal. In order to enable continuous audio playback for 
each channe l, the audio for each channel must be compressed so that 5 
lines of video per firame contain enough audio content for the playback 
time of one &ame of video (about 1/SOth of a second). 

The number of samples of audio required for each channel depends 
upon the desired sotmd quality. If lower sound quality is desired, fewer 
lines of video may be used, and vice versa. 

In the preferred embodiment five (5) lines of video are used per 
frame per channel of audio, with 105 samples per line at a sampling rate 
of 15.734 samples per second. TTie multiplexer (230) sequentlaUy 
samples each audio signal 15.734 times per second and altemaUvely 
directs 525 samples to each of the FIFO's (238. 242) associated with 
each audio channel. At a predetermined time before the audio fr^m each 
channel is to be Inserted into the video signal, the timing circuit (254), 
shown In Figure 9, instructs the fiaU FIFO for the channel to write the 
digital audio to the D/A converter (246) which converts it to a PAM 
signal. The timing circuit (254) then switches the video selector (258) 
to receive the PAM audio signal throu^ the buffer (250) as an input in 
Ueu of the video signal. In this manner, the additional audio channels are 
Inserted into the video. The standard audio remains unaffected in the 
normal audio carter. Once the audio has been Inserted into the video at 
the desired location, the timing circuit (254) commands the video select 
switch (258) to change its input back to the video signal. Hiis process is 
repeated for each audio signal at the video lines \diercln that audio signal 
is to be Inserted into the video signal. 

Topically, ten lines of the Vertical Blanking Interval (VBI) of the 
video signal are available for use in each field of video. In the preferred 
embodiment, for each frame of video, audio signals A and B are each 
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Inserted on five lines of the VBI of one field, and audio signals C and D are 
each Inserted on five lines of the VBI of the other field. Hie desired 
audio quality and number of audio channels dictates the ntimber of lines 
of video v^ch must be used to transmit the signals. Preferably, all of the 
5 additional audio channels can be inserted into the VBI. However, because 
of regulations regarding broadcast transmissionis and other uses of the 
video lines, if more~lines are- required for the audio services then active 
picture lines may have to be used. These active lines can be grouped and 
placed at the top or bottom of the picture which effectively will nor 

10 interfere with the viewed video due to the overscanning of most 
television sets. Altexnativefy. the lines can be randomly interspersed 
tfarou^out the program. 

In the preferred embodiment data is preferabfy sent on line 21 of 
the vertical blanking interval (VBI) using the predefined standard for 

15 transmitting data on line 21. The line 21 data is preferab^ sent as 16 
bits of serial data per field, and is placed into the video signal using a 
standard line 21 encoder. However, the data can altemattvefy be inserted 
cm other lines of the video. 

In an alternative embodiment, instead of inserting analog RAM 

20 samples as shown in Figure 9. multi-level digital samples, such as ADPCM 
samples may be inserted into the assigned VBI lines. In this 
embodiment a bandpass filter would filter the wideband analog audio in 
the desired firequency range. The signal is then sampled at some 
frequency rate in a sampler. Next, the RAM audio samples are quantized 

25 into one of n levels. An encoder then maps the amplitude levels selected 
by the quantizer into preferabty a three bit ADPCM samples. These 
samples are bufiiered and preferabty converted fipom binary to an 8-level 
converter. The ADPCM audio samples are then inserted in the video at 
the direction of a processor into the designated VBI lines. At the 

30 receiver, the corresponding extractor (174) sends the signal to a digital 
to analog converter and subsequently to a low pass filter to receive the 
filtered analog output This embodiment has the advantage of reducing 
end-to-end quantization noise and provides some degree of immunity to 
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channel noise and to noise resulting from VCR recording and playback of 
the video/analog signal. 

Several additional audio signals may be transmitted along with the 
Interactive video signal In other ways. An audio signal m^ be transmitted 
over each of the nonnal audio channels. The audio m^ be distributed in 
many ways using the available audio channels. For example, high quality 
background music for the interactive program could be transmitted over 
the nonnal audio channel. Then, lower quality audio for voice-overs could 
be Inserted onto the video signal and/or using the SAP signal. Preferably, 
the audio segments are aligned serial^ In the SAP channel. Hils would 
enable a larger number of audio signals to be transmitted with the 
Interactive program. In addition, the audio channels could also simply be 
combined with the video and data signals via a frequency division 
multiplexer. Hie audio channels can be digitized using any conmion 
analog-to-digital conversion technique such as PCM, DM, ADPCM. etc. 
Finally, vocoders (e.g., LPC, APC, channel vocoders) can be employed to 
reduce the requfred data rates and bandwidth requirements for 
transmitting voice. 

Hius, there are many variations In vMctx the present invention may 
be used to facilitate the transnilsslon of miiltlple audio channels. In one 
possible alternative embodiment. Instead of embedding the plurality of 
audio chazmels In the video signal, the channels are all sent as one audio 
signal using serial sequencing. Serial sequencing Is defined by reading 
the audio tracks from memoiy serially In sequence (l,e., Al, then A2, then 
A3, etc.). The resulting signal Is then sent as a secondaxy channel, or SAP 
diannel, with the primaiy channel containing the standard audio. In this 
embodiment these particular audio tracks woiild have to be sent prior to 
the time when the answers are schedtoled to be played to the subscribers. 
Hierefore, when the SAP channel Is received at the interactive program 
box (600). the signals will be independently decoded from the SAP 
channel, buffered and stored In memory (284) for later use when it is 
appropriate to playback the responses to the subscriber. The interactive 
program box (600) for this embodiment is described above and shown in 
figure 6. 
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Alternatively, the interactive system of the present invention can 
provide for transmission of the interactive signal by digital transmission 
means, resulting In reduced bandwidth requirements. In the digital 
transmission embodiment the interactive signal can be transmitted by not 
5 only satellite but ATM« Sonet, Tl or any other digital transmission 
system. 

— The digital transmission components^at the operations cente;r 
(608) are shown In Figure 10. As shown in Figure 10* the plurality of 
audio signals are ou^ut from a personal computer (114) or tape storage 

10 device and mixed in the audio mixer (118) with the conventional audio, 
preferably resulting in four output channels. Ihe audio signals are then 
preferably sampled, encoded and compressed in the audio digital 
encoder/compressor (500). The encoding technique can be a waveform 
coding technique such as PCM, ADPCM or DM. Alternatively, the signals 

15 can be encoded using synthesizer or vocoder techniques such as 
MUSICAM, Linear Predictive Coding (LPC), Adaptive Predictive Coding 
(APC)« and Sub*band coding. A single composite audio signal is output 
from the audio digital encoder/compressor (500). 

As shown in Figure 10, the video signal is input into the video 

20 switch (108) which outputs one of the video signals. The video digital 
encoder/compressor (504) takes the video signal, encodes and 
compresses the signal according to a known standard such as MFEG«1, 
MPEG-2, JPEG, or other DCT based coding scheme, wavelets, fractals or 
other transform based approaches. Altemattvelyt the video digital 

25 encoder/compressor (504), shown in Figure 10, could utilize a waveform 
based technique, such as PCM« ADPCM, or DM, for encoding and 
compressing the video signaL 

The data from the personal computer (114)* in Figure 10, is 
preferably compressed via a board in the computer (114) and delivered 

30 direcUy to the digital multiplexer (508). The digitized data, video and 
audio signals are delivered to the digital multiplexer (508) where the 
signals are buflfered and combined via time division mtiltiplexing, with 
appropriate synchronization signals applied, to form a composite sjntic 
serial data stream. At the output of the digital multiplexer (508), forward 
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error correction (FEC). the code depending on the selected transmission 
path, is applied by an FEC encoder (512). Hie composite Interactive 
digital signal is then transmitted to the receivers (158) via a suitable 
transmission means such as satellite, fiber optic, cable, DS, etc. 

In summary, the operations center (608) creates and transmits an 
interactive video signal by taking video, inserting multiple audio channels 
and data codes, and transmitting the resulting interactive video signal to 
the transmission system. 

E. The TransnHsfilnn Rv«fpm 

As shown in Figure 1, the system uses an Interactive program 
delivery system (98) -with any broadcast transmission means including 
satellite (150). cable (138) or television (142) to deliver the interactive 
program for distribution to the subscriber homes. Likewise, distribution 
of the signal at the receive end can be by any suitable transmission means 
(166) including a cable distribution system, fiber optic, etc. In a satellite 
broadcast transmission (150). the Interactive video signal is transmitted 
to uplink equipment (146) where it may be multipleiced. upconverted. 
modulated, amplified and transmitted by satellite (150) to a receiver site 
(154) for distribution to the classroom locations. In the analog 
environment, the interactive video signal enters an analog receiver (158) 
and then preferably a cable distribution system (166) that directs the 
signal to subscriber locations. In the alternative digital embodiment, the 
composite digital interactive signal enters a receiver (158) where it is 
demodulated and then preferably passed to a digital cable distribution 
system that directs the signal to the subscriber locations. Althoiigh a 
cable distribution system (166) is the preferred transmission media to 
the remote sites, the interactive video signal may also be distributed by 
any conventionally known technique, such as DBS. flberoptics. low or 
high power broadcast television, telephone lines, cellular networks, and 
similar technology can be used Interchangeably with this program 
delivery system. 

F. Return Path 

If desired, status messages and participant information can be sent 
back from the remote sites to the operations center (608) via return 
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paths. For example, subscriber selections could be stored In RAM (282) 
and later forwarded to the operations center (608) to aid in creating a 
subscriber profiles for targeted advertising. The return paths can be dial- 
up telephone lines or via an X.25 packet switched network, cable, 
wireless networks, etc. The preferred method of sending this 
information back is based on subscriber initiation. The alternative 
metibipd of sending this In formation I s b y polling the remote sites from 
the operations center computer (114). Alternatively, the remote sites 
can send performance information and messages after consolidation, at 
periodic intervals. 

If the Interactive program box (600) fimctions are performed at the 
cable headend as part of a multi^user ^tem, as described above, then the 
return path takes on greater importance. However, the path to and from 
the cable headend can be viewed simply as an extension of the signal path 
between the interactive program box (600) In the home and the 
television set (186). 

Input can be transmitted back to the operations center (608) in a 
two way cable system or in a telephone system via a modem (312). Tlie 
modem (312) is connected to the controller as shown in figures 3 and 6- 
8. The controller could send to the modem (312) a series of data 
streams characterizing the viewer's particular interests. 
III. System Opftratinn 

The simulcast system operates through the preparation, 
transmission and reception of a composite interactive program that can 
provide an interactive program to those subscribers' homes eqiilpped 
with an Interactive program box (600) and a regular television program 
for those homes not so equipped. As mentioned above, this dual 
fimctionality is provided by preferably embedding in primarily the 
nonvisual intervals of a standard video signal a plurality- of audio segments, 
graphics data and programming control data, the reception and 
processing of which provides a fully interactive experience for 
subscribers. Alternatively, the audio segments can be prestored at the 
Interactive program box (600). stored in external data storage (629). sent 
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down In serial fashion on the SAP or sent serially or in parallel on a 
separate channel. 

Information is stored on each audio segment. E^ch audio segment 
comprises messages In response to user selectlon(s) on the keypad (604). 
When the program begins, an INITIATION command Is sent from the 
broadcast or operations center (608) to all the Interactive program boxes 
(600). The television set (186) Is directed to default to a predetermined 
channel (e.g.. channel 1). 

Initially, the unauthorized and authorized interactive subscribers 
view the same standard television program with both viewing the 
standard video and hearing the standard audio. For nonlnteractlve 
subscribers, the standard video and audio is all they view and hear for the 
entire program. The interactive subscribers, on the other hand, are able 
to Interact with the program during designated time periods. 

At some point in time, trigger point codes are embedded in the 
composite interactive program signal, the trigger points Indicating the 
initiation of an interactive program. The VBI data codes immediately 
inform the interactive program box (600) that the program is Interactive. 
The controller (178) begins decoding the codes after detection. 
Graphics codes will also be sent as part of the signal. These codes enable 
the interactive program box processor (178) to read the graphics bit 
maps from memory (282. 286). corresponding to the codes, and direct 
the character generator (624) to overlay the graphics onto a 
predetermined portion of the screen. In this mamier. the first graphics 
interrogatory message can be displayed on the screen. 

While in the preferred embodiment and for purposes of illustration, 
the graphics messages are interrogatory, thQr may also be instructional, 
request user to select amongst several options to receive special features, 
provide status, or request subscriber profile Information from the 
subscriber. The graphics overlay may consist of simple alphammaeric 
characters forming a message or more sophisticated characters such as 
pictorial cartoon characters. While both the interactive and 
nonlnteractlve subscribers are hearing the standard audio, the Interactive 
viewers are requested through the use of the graphics overlay to enter a 
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choice on their keypads (604) or if the viewer is interacting via a 
personal computer (187), the keyboard connected to the personial 
computer (187), 

In this manner, at the next trigger point the controller directs the 
5 audio switch (620) to cut away from the standard audio and begin to play 
an introductory message such as ''Hi, welcome to the show, this is an 
interactive show. „Woidd you like, to interact with the show?" If the 
viewer desires to interact, he or she depresses the YES button on the 
remote, and the interactive show will begin. If the viewer does not wish 
10 to interact with the program, the controller directs the audio switch to 
branch back to the standard audio and the television will pick back up the 
conventional television program in progress. 

Simultaneously, a BEGIN INPUT command is sent to all of the 
interactive program box controllers (178) Instructing then to accept 
15 input from the infrared remotes (604). The last input before the 
expiration of the INPUT period is accepted as the subscriber's selection 
by the sjrstem. 

After the viewer has properly indicated their desire to interact with 
the program, preferably a set of graphics interrogatory messages will be 

20 presented one after another to solicit user selection useftil for obtaining 
desired viewer Information. For example, assume the show being 
presented is a show on redecorating a kitchen. First, an audio segment 
wlU be played stating "Hi, this is a show on redecorating your kitchen, 
are you Interested in redecorating yotir kitchen, YES or NO*? If the user 

25 selects the YES button on the remote* another audio segment may play 
nVell, thank you very much. Let me ask you a couple of quick questions 
on the dispU^.* At this point, the microprocessor (178) accesses a series 
of graphics Interrogatory messages stored in memory (282, 286) and 
directs the overlay chip (624) to display the graphics messages in 

30 predetermined order. Tlie user responses to these questions are used 
later in the program at particular trigger points for presenting interactive 
messages to the viewer. A series of sample Interrogatoiy messages may 
be displayed in succession as follows: 

-DO YOU HAVE WINDOWS IN YOUR KITCHEN? 
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A YES 
H NO.- 

-HOW BIG IS YOUR KITCHENr 

A. 10»20 B. 20^30 C. 30»40 D. 40*50 E. 50*60- 

"WHAT IS YOUR COLOR SCHEME? 

A. WHITE B. BROWN C. BLUE D. GREEN E. YELLOW" 
"WHAT TYPE OF FLOOR DO YOU HAVE? 
A WOOD B. FORMICA C. THE" 

Preferably, no audio accompanies the display of the graphics 
Interrogatory messages. The selections made by the user above are 
received by the Interactive program box (600). tagged with a code 
designating the particular related question and stored In memory (282. 
286) for later use at the appropriate time. When a trigger point Is 
Identified by the controller (178). the embedded data codes will identify 
the question corresponding to that trigger point. The controUer (178) 
will access the memory location address identified ^vlth the particular 
question and request the transfer of the particular user Input answer to 
that question. Hils user Input Is mapped via some algorithm Into an 
appropriate audio segment 

For example, suppose the narrator In the conventional program is 
currently discussing the size of the kitchen. At the next trigger point, 
the controller wlU access the memory location corresponding to the 
kitchen size question listed above. Assuming the user selected response 
-A 10*20". the controller may select and forward the following audio 
response to the audio switch. -Since you have a very smaU kitchen, may I 
suggest that you not put a dining room table in your kitchen." At the 
conclusion of this segment, the controller directs the controUer (178) to 
switch back to the standard audio. At the next trigger point, a similar 
sequence as described above wlU occur. In this manner, personalized 
Interactivity is provided at une3q)ected times to the subscriber. 

For unauthorized subscribers, the Interactive program box (600) 
acts similarly to a cable set top converter box. However, the Interactive 
program box (600) Is not required for recepUon of the conventional 
program by nonlnteractlve subscribers. If the subscriber has the box 
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(600) but is not authorized to Interact with the program for some reason, 
the video demodulator (616) simply demodulates the signal and the 
standard audio and video are transferred through the audio switch to the 
TV set (186) for presentation for the subscriber. 
5 Another application of the memory capability of the present 

invention Is targeted marketing of subscribers. In this application, the 
interacttye .program box processor compiles and stores Information 
including the Interactive program box ntmiber associated with the 
subscriber, time and date of the program viewing, the list of subscriber 
10 selecUons made by the subscriber during the program, and the program 
ID (contained In the program codes broadcast with the program). This 
data is prefnabfy stored together in data packets. From this data, a data 
code can be generated which provides an overall profile of the 
subscribers purchasing desires. This data code can either be sent via 
15 modem and over telephone lines to a retail location or displayed or 
printed out at the subscriber home. This code can then be used by the 
retail stores to oflfer unique purchasing price reductions and other 
options to the subscribers. 

While in the preferred embodiment the system has been described 
20 as a simulcast system whereby both an interactive program and a 
noninteractive program are broadcast to subscribers, the system does not 
require the program to be noninteractive. In other words, the system is 
not limited to providing the dual functional programs but can simply 
provide interactive programming, not receivable without the interactive 
25 program boxes (600), using the same elements described above. 

The above systems of the present Invention can be combined with 
those described in earlier Fteeman U.S. Patent Nos. 4.847,700. 
4,507.680. 4.573.072. 4.602.279. 4.364. 925. and 4.264. 924 and in 
commonly copending U.S. Patent i^pUcatlon Serial No. 07/797.298. filed 
30 November 25. 1991. resulting with home viewers equipped with a more 
sophisticated decoder box to receive more enhanced interactivity such as 
Instant replay, targeted advertising, etc.. through second or third 
transmitted video signals. aU of which can be provided with the 
personalized audio and graphics In the manner described above. 
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Using the foregoing embodiments, methods and processes, the 
ACTV simulcast system maximizes personalized attention and 
Interactivity to subscribers In their homes in real time using a low cost 
Interactive system. Although the present Invention has been described In 
detaU with respect to certain embodiments and examples, variations and 
modifications exist which are within the scope of the present Invention 
as defined In the foUowlng claims. 
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CLAIMS 

1. A fiiUy interactive system for receiving an interactive program signal 
simulcasting an Interactive program and a conventional television 
program, both programs provided with one conventional television video 
signal, to multiple subscriber locations permitting authorized interactive 
subscribers to Interact with and receive di£ferent and personalized 
responses firom the interactive program and-noninteractive subscribers to 
view and listen to a conventional television program, the system 
comprising: 

at least one means for receiving the interactive program 
signal comprising a plurality of audio channels, graphics data, 
control data, a standard video signal and standard audio signal; 
at least one means for displaying the standard video signal: 
at least one Interactive program means, connected to the 
receiving means and display means, comprising: 

a means for demodulating the standard video and 
standard audio signal; 

means for extracting, connected to the demodulating 
means, the plurality of audio channels and graphics and 
control data from the interactive . program signal, wherein 
each audio channel comprises one or more audio segments; 

means for storing the graphics and control data, 
wherein the control data comprises branching codes and the 
graphics comprises graphics codes designating at least one 
interrogatoiy message; 

means for gathering at least one subscriber entry; 

program selection means, operably cormected to the 
first storage means and extracting means, for reading the 
control data, controlling operation of the interactive program 
means, and selecting a personalized audio response 
comprising one or more audio segments, comprising: 

means for selecting one or more audio segments 

or graphics messages to form a personalized audio or 

graphics response, whereby the subscriber entry and 
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branching codes are xised in each branch in formulating 

the personalized audio or graphics response; 

a graphics generator, connected to the program 
selection means, for presenting graphics messages on the 
display means: and 

means for InteracUvely presenting the standard video 
and standard audio and the personalized audio response to 
the Interactive subscribers so that Interactive subscribers fully 
interact with the program to alter program content and 
Interact uniquely and differently from other interactive 
subscribers to receive personalized responses while 
nonlnteractive subscribers continue to receive a conventional 
television program. 

2. The fully Interactive system of dalm 1 wherein the plurality of audio 
channels, standard video signal and standard audio signal, and graphics 
and control data of the interactive program signal are analog and 
embedded in a standard NTSC video signal and the extracting means 
comprises means for removing the pluraUty of audio channels and 
graphics and control data from the standard NTSC video signal. 

3. Tlie fully Interactive system of claim 1 wherein the plurality of audio 
C5hannels. standard video signal and standard audio signal, and graphics 
and control data are digitally encoded and combined in a composite 
digital interactive signal and the receiving means ftirther comprises: 

means for demultiplexing the plurality of audio channels, 
standard video signal and standard audio signal, and data from the 
composite digital Interactive sigrud. 

4. The fully Interactive system of claim 1 wherein the Interactive 
program Is a recorded program to be received at a later time. 

5. The fully interactive system of claim 1 wherein the program 
selection means directs the storage means to store one or more previous 
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subscriber entries and wherein the program selection means comprises 
means for processing the previous subscriber entries along with the 
current subscriber entry so that a selection is based on prior subscriber 
entries, the cturrent subscriber entry and the codes in formulating the 
5 personalized audio or graphics response. 

- 6.- -The fully-lnteractlve-systcm . of -Claim -l ix^ereln the_program 
selection means directs the storage means to store one or more previous 
subscriber entries and \rtiereln the progjram selection means comprises 
10 means for processing the previous subscriber entries along with the 
current subscriber entry for determining the next graphics message to 
overlay onto the video signal. 

7. The fully interactive system of claim 1 wherein the means for 
15 gathering a subscriber entry comprises: 

a remote controller, comprising: 

means for entering subscriber endy; 

means for preparing a return signal to represent the 

subscriber entry; and 
20 means for transmitting the subscriber entry to the 

Interactive program box. 

8. The fully Interactive system of dalm 1 wherein each channel of the 
pltaraUty of channels comprises a sequence of audio response segments 

25 separated in time, with each audio response segment concurrent with 
audio segments In other channels thereby forming an array of audio 
response segments and the program selection means comprises means 
for picking an audio response segment firom the entire array of audio 
response segments so that the quantity of the selectable personalized 

30 audio response segments is greater than the quantity of the pliarality of 
audio channels. 

9. An interactive program production system used for creating a sln^e 
composite Interactive signal to multiple subscribers, the composite 
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Interactive signal simulcasting both a conventional television program and 
a fuUy Interactive program, both programs provided In a conventional 
television bandwidth to multiple subscribers, comprisliig: 

means for creating a conventlon£a video signal comprising a 
5 standard audio and standard video; 

means for generating graphics data, the data being used for 
identifying graphics characters that are to be overlaid on the video 
at predetexmined points in time; 

means for providing a plurality of audio channels, the audio 
10 channels comprising audio segments, the audio segments being of 

substantially equal length in time and related in content to each 
other: 

means for creating control data comprising codes, whereby 
the codes are used for initiating, controlling, and branching 
15 between the plurality of audio channels: 

means for combining the conventional video signal, control 
data and plurality of audio channels into a single composite 
Interactive signal: and 

means for transmitting the single composite interactive signal 
20 to multiple subscribers. 

10. The Interactive program production system of Haim 9, the imit 
further comprising: 

means for dlgltallyr encoding the conventional video signal and 
25 plurality of audio rhgnti^i s; and 

means for multiplexing the control and graphics data, the 
plurality of audio channels and the conventional video signal Into a 
single composite digital interactive signal. 

30 11. The Interactive program production s]rstem of claim 9 wherein the 
conventional video signal, graphics and control data and plurality of audio 
channels are analog and the combining means comprises means for 
embedding the graphics and control data, pliorality of audio chaimels and 
conventional video signal Into a standard NTSC video signal. 
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12* Hie Interactive program production system of claim 9 wherein the 
conventional video signal comprises a vertical blanking interval and the 
embedding means comprises means for inserting the graphics and 
5 control data and pltirallty of audio channels into the vertical blanking 
interval of the conventional video signal* 

13. The interactive program production system of claim 9 wherein the 
transmission means comprises a television broadcasting means. 

14. Hie interactive program production system of claim 9 wherein the 
transmission means comprises a cable means. 

15. The interactive program production system of claim 9 wherein the 
15 transmission means comprises a satellite broadcast means. 

16. The interactive program production system of daim 9 wherein the 
means for providing a pltirality of audio channels comprises a means for 
linking a sequence of audio response segments in each channel forming 

20 an array of stacked audio segments so that the quantity of selectable audio 
response segments is greater than the quantity' of the plurality of audio 
channels. 

17. The interactive program production system of daim 9 further 
25 comprising a means for recording the composite interactive signal for 

- later transmission. 

18. Tlie interactive program production system of daim 9 wherein the 
composite interactive signal comprises a Uve program. 

30 

19. An interactive program production system used for creating a single 
composite interactive signal to mtiltiple subscribers, the composite 
interactive signal simulcasting both a conventional television program and 
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a ftiUy Interactive program, both programs provided In a conventional 
television bandwidth to multiple subscribers, comprising: 

means for creating a conventional video signal comprising a 
standard audio and standard video; 

means for generating graphics data, the data being used for 
identifying graphics characters that are to be overlaid on the video 
at predetermined points in time including means for generating a 
first graphic Interrogatory question to overlay the conventional 
video signal if the fuDy interactive program is being simulcast: 

means for providing a plurality of graphic channels, the 
graphic channels comprising graphic segments responsive to the 
first graphic interrogatory, the graphic segments being of 
substantially equal length in time and related in content to each 
other; 

means for creating control data comprising codes, whereby 
the codes are used for initiating, controlling, and branching 
between the plurality of audio and graphic channels; 

means for combining the conventional video signal, control 
data and plurality of graphic channels into a single composite 
Interactive signal; and 

means for transmitting the single composite interactive signal 
to multiple subscribers. 

20. The interactive program production system of Claim 19 wherein 
the system is used to simulcast a sporting event in a conventional and an 
Interactive format, the means for generating a first graphic interrogatory 
being used to enable selection by an interactive subscriber of statistics for 
a particular player or team and the means for generating a plurality of 
graphic channels being used to provide the statistics selected in response 
to the interrogatory question. 

21. Hie Interactive program production system of claim 19 further 
comprising: 
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means for providing a plurality of audio channels, the audio 
rh^^nn^lff comprising audio segments related in time and content to 
the plurality of graphic channels, the audio segments being of 
substantially equal length in time and related in content to each 
5 other: and i)(^erein the combining means comprises means for 

combining the conventional video signal, control data and plurality 
of-audio and-graphlc channels-lnto a-slngle composite interactive 

signed. 

10 22. An interactive program production system used for creating a single 
composite interactive signal to multiple subscribers, the composite 
intexactive signal simulcasting both a conventional television program and 
a ftally interactive program, both programs provided in a conventional 
television bandwidth, to multiple subscribers, comprising: 
15 means for creating a conventional video signal comprising a 

standard audio and standard video: 

means for generating graphics data, the data being used for 
identifying graphics characters that are to be overlaid on the video 
at predetermined points in time Including means for generating a 
20 first graphic Interrogatory question to overlay the conventional 

video signal if the fully interactive program is being simulcast: 

means for providing a plurality of graphic channels, the 
graphic channels comprising graphic segments responsive to the 
first graphic interrogatory, the graphic segments being of 
25 substantially eqtial length in time and related in content to each 

other: 

means for providing a plurality of audio channels, the audio 
channels comprising audio segments related in time and content to 
the plurality of graphic channels, the audio segments being of 
30 substantially equal length in time and related in content to each 

other: 

means for creating control data comprising codes, whereby 
the codes are used for initiating, controlling, and branching 
between the plurality of audio and graphic chaimels: 
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means for combining the conventional video signal, control 
data and plurality of audio and graphic channels into a single 
composite interactive signal; and 

means for transmitting the single composite Interactive signal 
to multiple subscribers. 

23. The Interactive program production system of Claim 22 wherein 
the system is used to simulcast a sporting event in a conventional and an 
interactive format the means for generating a first graphic inteirogatoiy 
being used to enable selection by an interactive subscriber of statistics for 
a particular player or team: the means for generating a plurality of graphic 
channels being used to provide the statistics selected in response to the 
interrogatoiy question: and the means for providing a plurality of audio 
channels being used to provide a voice response reading or commenting 
on the statistics chosen wblcb can be the voice of the player or owner of 
the team whose statistics were requested in the response to the 
Interrogatoiy question. 

24. An interactive program production system used for creating a single 
composite interactive signal to multiple subscribers, the composite 
interactive signal simulcasting both a conventional television program and 
a fully interactive program, both programs provided in a conventional 
television bandwidth to multiple subscribers, comprising: 

means for creating a conventional video signal comprising a 
standard audio and standard video: 

means for generating graphics data, the data being used for 
identifying graphics characters that are to be overlaid on the video 
at predetennined points in time; 

means for providing a plurality of interactive video channels, 
the audio channels comprising video segments, the video segments 
being of substantially equal length in time and related in content to 
each other: 
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means for creating control data comprising codes, i^ereby 
the codes are used for initiating, controlling, and branching 
between the plurality of interactive video channels; 

means for combining the conventional video signal, control 
data and plurality of interactive video chaxmels into a single 
composite interactive signal; and 

means for^transmitting the single composite interactive s 
to multiple subscribers, each of the subscribers having a means to 
receive the conventional video signal and at least one of the 
subscribers having means to receive the interactive video signal 
vAih the ability to switch between the plurality of video channels. 

25. The interactive program production system of Claim 24 wherein 
the system is used to simulcast a sporting event In a conventional and an 
interactive format wherein the means for generating graphics data 
includes means for generating a first graphic Interrogatory question to 
overlay the conventional video signal if the fully interactive program is 
being simulcast to enable selection by an interactive subscriber of 
different angles of play, a close-up of a particular player or team or a 
replay of the last play and the means for generating a plurality of 
interactive video chaxmels is used to provide the video chaxmel which 
displays the selection selected in response to the interrogatory question. 

26. A method for offering a single composite interactive program signal 
for simulcasting an interactive program and a conventional program both 
within a conventional television bandwidth to multiple subscriber homes, 
the method comprising the steps of: 

packaging the single composite Interactive program signal 
comprising a plurality of audio chaxmels, graphics data and control 
data and a standard video signal and standard audio signal, wherein 
the single composite interactive program signal comprises a 
conventional television program and an interactive program; 

transmitting the single composite Interactive program signal 
to at least one receive site location; 
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recetving the single composite Interactive program signal: 

extracting the audio channels, graphics data and control data 
from the single composite Interactive program signal; 

storing the graphics data and control data, whereby the 
control data comprises branch codes: 

generating graphics characters, the graphics characters 
comprising at least one Interrogatoiy message: 

overlaying the graphics characters onto porUons of the 
standard video signal to obtain a resulting video signal: 

displaying the resulting video signal on a display monitor; 

gathering a subscriber entry to the graphics interrogatory 
message: 

branching between the plurality of audio channels to form a 
personalized audio response, whereby the subscriber entry and 
branching codes are used in formulating the personalized audio 
response: 

presenting the personalized response to the subscriber, 
whereby each subscriber interacts with the interactive program 
Individually and differently from other subscribers, with each 
subscriber receiving a unique personalized audio response. 

27. An interacUve home entertainment and educational system for 
receiving a fully interactive program signal, the fiiUy interactive program 
signal transmitted in a single standard television broadcast bandwidth, 
the system comprising: 

at least one means for receiving the Interactive program 
signal comprising a pluraUty of audio channels comprising audio 
segments, data and a video signal, the interactive program signal 
carried in a single standard television broadcast bandwidth: 
at least one means for displaying the video signal; 
at least one interactive program box, connected to the 
receiving and display means, comprising: 

means for extracting, connected to the receiving 
means, the plurality of audio nhgntT^is and data; 
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means for storing a plurality of graphics Interrogatory 
messages: 

means for gathering a subscriber entxy: 
a program selection means, connected to the esctractlng 
5 means and storage means, for selecting a personalized 

graphics interrogatory messages from the pliarality of graphics 
interrogatory messages and an audio response comprising at 
least one personalized audio segment from the plurality of 
audio channels, comprising: 
10 means for processing previous siobscriber entries 

along with the current subscriber entry for determining 
a graphics interrogatory message; 

means for formulating a personalized audio 
response comprising at least one audio segment, 
15 ix^ereln the audio response is chosen as a function of a 

most current subscriber entry and at least one previous 
subscriber entry; 

a graphics generator, cormected to the program 
selection means, for overlaying the selected graphics 
20 Interrogatory message onto the video signal; and 

a means for presenting the personalized audio response 
to the subscribers thereby allowing each subscriber to fully 
interact uniquety and di£ferentty frt)m other subscribers at 
other subscriber locations. 

25 

28. The interactive home entertainment and educational system of 
claim 27 wherein the plurality of audio charmels, video signal, and data of 
the interactive program signal are analog and embedded in a standard 
NTSC video signal and the extracting means comprises means for 

30 removing the plurality of audio charmels and graphics and control data 
from the standard NTSC video signal. 

29. The interactive home entertainment and educational system of 
Hflim 27 wherein the plurality of audio charmels, video signal, and data 
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are dIgltaUy encoded and combined In a composite digital Interactive 
signal and the receiving means further comprises: 

means for demultiplexing the plurality of audio channels. 

video signal and data from the composite digital interactive signal; 

and 

means for converting the demultlplexfed audio channels and 
the demultiplexed video signal Into analog signals. 

30. The Interactive home entertainment and educational system of 
claim 27 wherein each channel of the plurality of audio channels 
comprises a sequence of audio response segments separated in time. 
vnth each audio response segment concurrent with audio segments In 
other channels thereby forming an aixay of audio response segments and 
the program selection means comprises means for picking an audio 
response segment from the entire array of audio response segments so 
that the quantity of the selectable personalized audio response segments 
Is greater than the quantity of the plurality of audio channels. 
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